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TITLE OF THE INVENTION 
METHOD AND APPARATUS FOR COMPENSATING FOR MOTION PREDICTION 



5 BACKGROUND OF THE INVENTION 

Technical Field of the Invention 

This invention relates to a method and apparatus for 
compensating f or motionprediction, which is suitably applicable 
to an image coding apparatus for transmitting, for example, 
10 motion picture signals over networks, such as the Internet. 
Description of the Related Art 

An image coding apparatus, conventionally, is configured 
to digitize amotion image signal supplied from an external source 
and then carries out code processing on it conforming to a 
15 predetermined image-coding scheme, thereby generating 
image -compressed information. 

The foregoing image -coding scheme is a known image coding 
scheme, called MPEG, standardized for the purpose of coding a 
general -use image by the Moving Picture Experts Group (MPEG) 
20 of ISO/IEC, and the image-coding scheme, called H.26* (i.e., 
H.261, H.262, ...) , standardized aiming at coding a 
television-conference image . 

Meanwhile, the recent widespread use of personal digital 
assistants (PDA's) including cellular phones gives rise to the 
25 need for an image-coding scheme to realize still higher coding 
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efficiency. In order to cope with this, the standardization 
of another image-coding scheme, called JVT (Joint Model of 
Enhanced-Compression Video Coding) (hereinafter, this is called 
a JVT coding scheme) , is currently being pushed forward by the 
5 MPEG and ITU entities. 

Under the JVT coding scheme, when carrying out 
motion-prediction compensation, mot ion vectors can be searched 
over the macro-block having vertically horizontally 16x16 pixels 
with the use of four kinds of size patterns (hereinafter, this 

10 is referred to as a macro-block mode) TPl - TP4 , e.g. four sets 
of motion compensating blocks comprising one set of pixel blocks 
having a size of vertical/horizontal 16x16 pixels (hereinafter, 
this is referred to as motion compensating blocks) , two sets 
of motion compensating blocks each having a size of 

15 vertical /horizontal 8x16 pixels , two sets of motion compensating 
blocks each having a size of vertical/horizontal 16x8 pixels, 
or four sets of motion compensating blocks each having a size 
of vertical/horizontal 8x8 pixels, as shown in Fig. 16. Each 
of the motion compensating blocks related to these macro-block 

20 modes TPl - TP4 can independently have the motion vectors. 

Furthermore, concerning the four sets of motion 
compensating blocks related to the macro-block mode TP4 , motion 
vectors can be searchedoneachof the four sets of the compensating 
blocks by use of four kinds of sub-size patterns (hereinafter, 

2 5 this is referred to as a sub-macro block mode) TPS - TPS, e.g. 
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one set of motion compensating blocks having a size of 
vertical/horizontal 8x8 pixels, two sets of motion compensating 
blocks each having a size of vertical/horizontal 4x8 pixels, 
two sets of motion compensating blocks each having a size of 
5 vertical/horizontal 8x4 pixels, or four sets of motion 
compensating blocks each having a size of vertical/horizontal 
4x4 pixels. Each of the motion compensating blocks related to 
these sub-macro block modes can independently have the motion 
vectors . 

10 Accordingly, under the JVT coding scheme, one macro-block 

can have 16 motion vectors in maximum (see, for example. Draft 
ISO/IEC 14496 - 10:20 02 (e)), the entire contents of which being 
incorporated herein by reference - 

Meanwhile, under the JVT coding scheme , when compensating 

15 for motion prediction, it is possible to hold a group SF of a 
plurality of reference frame images. It is possible to carry 
out a block matching by using two or more reference frame images 
SFl, SF2 and SFn-1 existing prior (past) to an objective frame 
image OF to be code-processed, a block matching by using two 

2 0 or more reference frame images (not shown) existing subsequent 
(future) to an objective frame image OF, or a block matching 
by using two or more reference frame images (not shown) existing 
at around an objective frame image OF. The group SF of reference 
frame images held as the referential candidates is referred to 

25 as a Multiple Reference Frame (see, for example. Draft ISO/IEC 
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14496 - 10:2002 (e)). Furthermore, the JVT coding scheme 
adopts optimized motion-prediction compensation called 
Rate-Distortion Optimization (hereinafter, this is referred to 
as RD optimization) . By RD optimization, all the macro-blocks 
5 of an objective frame image OF can be block-matched with the 
group SF of a plurality of reference frame images (Fig. 16) , 
thereby determining a difference value (distortion) in a 
difference absolute -value sum or the like. Taking into account 
a quantizing function (Lagrange multiplier for motion vectors) 

10 and generation code amount of motion vector difference, a motion 
vector is searched for which can generate a minimum code amount, 
thereby enhancing the coding efficiency for motion vectors See 
DRAFT ISO/IEC 14496-10: 2002(E), and 

Rate-Distortion Optimization for Video Compression (G. 

15 Sullivan and T. Wiegand, IEEE Signal Processing Magazine, 
1998-11) , the entire contents of which being incorporated herein 
by reference. 

Meanwhile, in the coding apparatus employing the 
foregoing JVT coding scheme, while all the macro-blocks are 

20 changed to motion compensating blocks in various pixel -based 
sizes (Fig. 16) between the macro-block entirety and a group 
SF of a plurality of reference frame images, reference is made 
to a generation code amount table representative of a generated 
code amount of motion vector difference and further a quantizing 

25 function (Lagrange multiplier for motion vectors) is determined 
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besides a difference value (distortion) , thereby carrying out 
a motion-prediction compensation. Thus, the present inventors 
have recognized that there is a drawback that process burden 
is increased due to code processing . 

5 

SUMMARY OF THE INVENTION 
It is an object of the present invention to provide a method 
and apparatus for compensating for motion prediction which can 
reduce the process burden due to code processing while possibly 
10 avoiding the lowering image quality. 

In order to solve the foregoing and other problems , in the present 
invention, when compensating for motion predict ion on each mot ion 
compensating blocks formed by dividing an objective frame image 

15 of successive frame images by using a plurality of reference 
frame images while sequentially changing pixel -based sizes of 
the motion compensating blocks. Pixels are thinned out of the 
motion compensating block having a greatest one of the 
pixel -based size to be taken as an uppermost layer of among the 

20 pixel-based sizes, to generate a size-reduced block in a lower 
layer having a predetermined size- reduc t ion rat io . Thereafter , 
motion vector search ranges are determined respectively within 
the reference frame images, on the basis of the size-reduced 
blocks and a plurality of size-reduced reference images reduced 

25 in size correspondingly respectively to the size-reduction 
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ratios of the size-reduced blocks . An optimal one of the motion 
vectors is detected while sequentially changing the pixel -based 
sizes of the motion compensating blocks by using the search range 
determined . 

5 In this case, motion vectors can be detected only in a 

limited search range of each reference frame image without using 
the entire of the reference frame images. 

BRIEF DESCRIPTION OF THE DRAWINGS 
10 Fig. 1 is a schematic diagram for explaining the motion 

compensation at 1/4 -pixel accuracy under the JVT coding scheme; 

Fig. 2 is a schematic diagram for explaining the motion 
compensation at 1/8 -pixel accuracy under the JVT coding schemer- 
Fig. 3 is a schematic diagram showing the motion 
15 compensation at 1/8-pixel accuracy using pixel-based values on 
a diagonal line; 

Fig. 4 is a schematic diagram for explaining the motion 
compensation in chrominance signals; 

Fig. 5 is a block diagram showing a configuration of an 
2 0 image coding apparatus according to the present invention; 

Fig. 6 is a schematic diagram for explaining a 
thinning-out process on a field; 

Fig. 7 is a block diagram showing a configuration of a 
motion-prediction compensating section; 
25 Fig. 8 is a schematic diagram showing a manner of 
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hierarchi zat ion ; 

Fig. 9 is a flowchart showing a motion-prediction 
compensating process procedure; 

Fig. 10 is a schematic diagram for explaining an example 
5 to search for a motion vector; 

Fig. 11 is a schematic diagram for explaining a first 
determining example on a motion vector search range; 

Fig. 12 is a schematic diagram for explaining a detecting 
sequence of motion vectors; 
10 Fig. 13 is a schematic diagram for explaining a second 

determining example on a motion vector search range; 

Fig. 14 is a schematic diagram for explaining a motion 
vector detecting sequence in another embodiment; 

Fig. 15 is a schematic diagram for explaining a block 
15 matching according to another embodiment of the present 
invention; 

Fig. 16 is a schematic diagram showing a motion 
compensating block as defined under the JVT coding scheme; and 
Fig. 17 is a schematic diagram for explaining a Multiple 
2 0 Reference Frame as defined under the JVT coding scheme. 

DETAILED DESCRIPTION OF THE INVENTION 
JVT Coding Scheme Overview 

At the outset, explanation is made on the motion 
25 compensation as defined under the JVT coding scheme. Under the 
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JVT coding scheme, motion compensation is available at 1/4 -pixel 
accuracy and at 1/8 -pixel accuracy. The motion compensation 
process at 1/4 -pixel accuracy (interpolation process) is 
explained with reference to Fig. 1. 
5 The JVT coding scheme uses a FIR (Finite Impulse Response) 

filter as a band limiting filter, to compensate for motion. As 
a filter coefficient for generating a pixel -based value 
(luminance signal) at 1/2 -pixel accuracy, there is defined a 
6-tap filter coefficients as follows. 




(1) 
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Herein, as for the motion compensation (interpolation) 
for the pixel -based value b, h shown in Fig. 1, sum-of -products 
operation is made by the use of the filter coefficients of 
vector(l), according to the following equation. 

b = (E-5F+2 0G+2 0H-5H + J) 

h-(A-5C+20G+20M~5R + T) (2) 

15 

Then, provided that the 8 bits of from "0" to "255" are considered 
as Clip 1, computation is to be made by a clip process to remove 
the other part than Clip 1 as given in the following equation. 

b = Clipl((b+16)>>5) 

h = Clipl((h+16)>>5) (3) 

20 In this connection, ">>5" in Equation (3) means to shift 
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by 5 bits, wherein actually division is in an amount of 2^, 
Meanwhile, "+16" in Equation (3) is in order not to remove 
fractions (not to be rounded) when division is made in an amount 
of 2^ 

5 Meanwhile, concerning the pixel -based value j, a 

pixel -based values aa, bb, cc, dd, ee, ff, gg, hh are generated 
similarly to the foregoing pixel -based value b, h by the use 
of the filter coefficients of Vector (1) . Thereafter, using the 
filter coefficients of Vector (1) , sum-of -products operation is 
10 made according to the following equation 

j = cc-5dd + 20h+20m-5ee+ff (4) 

or the following equation 
j=aa-5bb + 20b+20s'-5gg + hh (5) 

Thereafter, a clip process is made according to the following 
15 equation, thus carrying out computation, 

j = C 1 i P 1 C ( j + 5 1 2 ) > > 1 0 ) ( 6 ) 

Meanwhile, the pixel -based values a, c, d, n, f, i, k, 
q can be computed by a linear interpolation with a pixel -based 
values at integer-pixel accuracy (corresponding to the hatching 
20 in the figure) and a pixel-based value G, H, UBl, h, j , m, s 
at 1/2 -pixel accuracy, as given in the following equation. 
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a = (G + b) > > 1 



c = (H + b) > > 1 



d= (G + h: 



> > 1 



n = (MH- h) > > 1 



f = (b + j ) > > 1 



i = ( h + j ) > > 1 



k = ( j +m) > > 1 



q = ( j + s ) > > 1 



(7) 



Meanwhile, the pixel -based values e, g, p, can be computed 
by a linear interpolation using only the pixel -based values b, 
h, m, s at 1/2 -pixel accuracy, as given in the following equation. 



e = (b + h) > > 1 



g = (b + m) > > 1 



p = (h+ s) >> 1 



(8) 



Furthermore, the pixel -based value r is a point called 
"interpolatively challenging position" under the JVT coding 
scheme, which can be computed as in the following equation. 



(G + H + M+N+2) >>2 



(9) 
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Now, the motion compensating process at 1/8 -pixel accuracy 
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(interpolation process) is explained with using Fig. 2. 

Under the JVT coding scheme, the following equation is 
defined as . a polyphenol filter coefficient to generate a 
pixel -based value (luminance signal) at 1/4 -pixel accuracy. 

(-3. 12. - 37. 229. 71. - 21. 6. -1) 
(-3. 12.-39. 158. 158.-39. 12.-3) 
6. - 21. 71. 2 29. - 37. 12. - 3) 

(10) 

5 

Concerning the pixel -based values b^, b^ at 1/4 -pixel 
accuracy, the filter coefficients in Equation (10) are used to 
make a sum-of -products operation with a pixel -based value A at 
integer-pixel accuracy as an input sample. Thereafter, a clip 
10 process is made according to the following equation, thereby 
carrying out computation. 

b^' ^ =C 1 i p 1 (b + 1 2 8) >> 8 (11) 

Meanwhile, concerning the pixel -based value c™, c*^, a 
sum-of -products operation is first carried out on the pixel -based 
15 value b^, b"^. Then, using a value not to make a clip process, 
a sum-of -products operation is further made with the pixel -based 
value b^, b^ as an input. Thereafter, a clip process is made 
according to the following equation, thereby carrying out 
computation. 
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c«'<'=Clipl(c + 32768)>>16 (12) 

Meanwhile, the pixel -based value d at 1/8 -pixel accuracy- 
can be computed depending upon its position by a linear 
interpolation according to the following equation. 

d= (A+b" ) >> 1 
d = (A + bM > > 1 
d = (b" + ) >>1 
d= (b" + c" ) >>1 
d= (C + 0" ) >>1 
d= (b^ + b" ) >> 1 

d= (c" + C'" ) >>1 03) 

Meanwhile, the pixel -based value e can be computed by using 
the pixel-based value b"^, b'^ positioned on a diagonal line, 
according to the following equation. 

e = (b " + b " ) > > 1 Cl4) 

Meanwhile, the pixel -based value g can be computed 
according to the following equation. 

g=(A+3c'"+2)>>2 (15) 

Furthermore, the pixel -based values f^, f^ can be computed 



12 



by the following equations. 



f^ = C3b'+b''+2)>>2 

f ^' = ( 3 b " + b ' + 2 ) > > 2 (16) 

Herein, Fig. 3 shows a concept figure on the technique 
to generate a pixel -based value at 1/8 -pixel accuracy by linear 
interpolation using a pixel -based value existing on the diagonal 
line . 

Now, explained is the motion compensating process 
(interpolation process) of a chrominance signal under the JVT 
coding scheme, by using Fig. 4. Under the JVT coding scheme, 
in both cases of compensation for a motion at 1/4 -pixel accuracy, 
and motion at 1/8 -pixel accuracy, the chrominance signal at 
decimal -pixel accuracy can be computed by linear interpolation 
according to the following equation. 

V = 

(s-d^ )(s-d ' )A+d - (s-d- )B + (s-d- )d +d ^ d ^ D +sV2 

(17) 

Meanwhile, there are a number of motion compensating 
processes defined under the JVT scheme, as described above. It 
is important, in generating image compression information at 
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high compression ratio, to select an optimal one of the relevant 
modes for the macro-block, i.e. , to search for a motion vector . 
For this reason, a motion vector is searched for by RD 
optimization. 

Now, such RD optimization is explained in detail. 

In RD optimization, provided that the search result is 
J (m, ^motion) # the motion vector is m = (mx, my)"^, the predictive 
motion vector is SA(T)D(s, c (m) ) , p = (px/ Py)"^, the Lagrange 
multiplier for the motion vector is ^motion/ and the generated 
information amount of a motion vector difference determined by 
a generation code amount table is R (m - p) , then the motion search 
at every pixel accuracy in motion compensation can be determined 
according to the following equation. 

J (m. Amotion) 

= SA(T)D(S, C (m) ) + Amotion • R (m- p) 

(18) 

Herein, under the JVT coding scheme, there are defined 
two kinds of entropy coding, i.e., the method based on UVLC 
(Universal Variable Length Coding) and the method based on CABAC 
(Context -based Adaptive Binary Arithmetic Coding) . For the 
generated information amount, ULVC is applied even where CABAC 
is being applied. The residual energy (distortion) part of 
motion vector (corresponding to SA (T) D (s, c (m) ) in EquationlS ) 
can be determined by the following equation, provided that the 
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current -frame image signal is s and the reference- frame image 
signal is c. 

S A Dc>( s , c (m) ) = 

l'' Is Cx, y] -c Cx-mx . y-niy ] I (19) 

Note that, when correcting a motion vector at half -pixel 
5 accuracy or lower, SATD is used because determination is possible 
by the use of a Hadamard transform instead of a discrete cosine 
transform. 

The Lagrange multiplier (X^motion) # provided that the 
quantization parameter is QP, for the I and P frames is given 
10 as in the following equation. 

,>= |/~o7T7*T«^ (20) 

Meanwhile, for the B frame is given as in the following 
equation . 

^MODE.B =V4*0.85*2^'^ . ....(21) 

15 

Now, explained is the case of selecting a reference frame 
image to be actually referred from a group SF of a plurality 
of reference frame images (Fig. 17) in RD optimization. In RD 
optimization, provided that, concerning selecting a reference 
20 frame image, the result of that selection is J(REF| X-motion) and 
the value the amount of generated information on the reference 
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frame is determined by UVLC is R(REF) , selected is a reference 
frame image having a value at which the relevant selection result 
(J(REF| Amotion ) ) assumes a minimum according to the following 
equation. 

J (R E F/^ A motion) 

-SATD (s, c CREF, m (REF) 

+ Amotiox- (R (m (REF) -p (REF) ) +R (REF) ) 

(22) 

5 

Meanwhile, concerning to select a predictive direction 
of N X M blocks in the B frame, provided that the result of that 
selection is J (PDIR| A^motion) / selected is a predictive direction 
of N X M blocks having a value at which the relevant selection 
10 result (J(PDIR| A^motion) ) assumes a minimum according to the 
following equation . 

J (PD I RX 1 motion) 

= SATD(s, c (PDIR, m(PDIR)) 
+ 1 MOTION (R (m (PD I R) - p (PD I R)) 
+ R (REF (PD I R))) 

(23) 

Now, explained is macro-block mode selection mentioned 
above in Fig. 16, in RD optimization. In RD optimization. 
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provided that, concerning selecting a macro-block mode, the 
result of that selection is J(s, c, MODE| OP, ?Imotion) / selected 
is a macro-block mode having a value at which the relevant 
selection result (J(s, c, MODE] OP, ^motion) ) assumes a minimum 
5 according to the following equation. 



J(s,cMODE\QP,Zmotion) 
= SSD{s,cMODE\QP) 

+ KonoN • R(s,c,MODE \ QP) (24) 

10 

The MODE, to assume a selection candidate for the 
macro-block mode, is shown based on each frame type in Table 
1 . 



I f r a m e : M 



O D E G { INTRA4X4, INTRA16xi6. } 



r INTRA4X4, INTRA16X16, SKIP, >, 
P f r am e : MODEe \ \ 

^ 16x16,16x8.8x16,8x8 ^ 



. INTRA4x4, INTRA16xl6, DIRECT, >, 
B f r ame : MODEe \ \ 

^ 16x16, 16x8, 8x16, 8x8 ^ 



15 Table 1 
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In Table 1, "SKIP" has a size of vertical/horizontal 16x16 
pixels and represents vector residues and those having 
coefficient residues not to be sent toward the decode side . "SSD" 
is a value of error sum of squares represented by the following 
5 equation provided that the macro-block generated information 
amount when MODE and QP are selected is R(s, c, MODE| QP) , the 
luminance components of the re-configured and original images 
are Cy [x, y, MODE] QP and Sy [x, y] , and the chrominance components 
are Cu/ Cv and s^, Sv 

SSDCs, c, MODE|QP) = 

"z" (Sv «&x, y] -c, Cx, y, MODE I QP] ) ' + 

X = 1 . y - t 

(su Cx, y] -Cu Cx. y. MODE I QPD ) ' + 

X . 1 . y = I 

s\sv Cx. y] -Cv Cx. y, MODE I QP] ) ' 

X 1 . y = I 

(25) 

10 

Note that the amount of generated information includes 
that corresponding to all the pieces of information, such as 
headers, motion vectors and orthogonal transform coefficients . 
Meanwhile, the Lagrange multiplier (^motion) is provided as in 
15 the Equations (20) and (21) . 

The sub-macro block mode mentioned in Fig. 16 is 
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selection-processed similarly to the case on the macro-block 
mode . The MODE , to assume a selection candidate for the sub-macro 
block mode, is shown based on each frame type in Table 2. 

. INTRA4Ax4, 
P f r ame : MODE^ \ 

^ 8x8, 8x4, 4x8, 4x4 

. I NTRA4x4, DIRECT, 

B f r ame : MODE^ \ \ 

^ 8x8, 8x4, 4x8, 4x4 ^ 

5 Table 2 

In case the foregoing processes as defined under the JVT 
coding scheme be applied to the image coding apparatus, a huge 
amount of operation is needed upon searching for motion vectors . 
10 Accordingly, the present invention reduces the amount of 
operation possibly without losing the compression efficiency. 

Now, one embodiment of the present invention is described 
in detail with reference to the drawings. 

15 Image Coding apparatus Configuration 

Fig. 5 is a block diagram of the overall configuration 
of an image coding apparatus 1 to which the JVT coding scheme 
is applied. 

In the image coding apparatus 1, the motion image signal 
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SI supplied from an external source is generated into motion 
image data Dl through an A/D converting section 2, followed by 
being temporarily stored in an image rearranging buffer 3 . In 
the image rearranging buffer 3, the motion image data Dl are 
5 rearranged in a compression-coding order on a frame- image unit 
basis. The rearranged frame images are sequentially forwarded 
as frame data D2 to an adder 4 and motion-prediction compensating 
section 5. 

The motion-prediction compensating section 5, although 

10 detailed later, divides the frame image of the frame data D2 
provided from the image rearranging buffer 3 into motion 
compensating blocks having a size of vertical/horizontal 16x16 
pixels, the greatest pixel -based size of the motion- compensating 
blocks mentioned in Fig. 16 (hereinafter, this is referred to 

15 as the maximum mot ion- compensating block) . Based on each 
maximum motion-compensating block, optimal motion vectors D3 
are respectively detected by using a plurality of reference frame 
images while sequentially changing the pixel -based size of the 
motion- compensating block. These are sequentially forwarded 

20 to a reversible coding apparatus 18. 

Meanwhile, the motion-prediction compensating section 5 
sequentially forwards to the adders 4 and 11 a plurality of 
predictive data D4 obtained by compensating for motion according 
to a plurality of motion vectors D3 . 

25 In the case that predictive data D4 is provided from the 
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motion-prediction compensating section 5, the adder 4 subtracts 
a plurality of predictive data D4 corresponding to the relevant 
frame data D2 from the respective frame data D2 (image type other 
than I picture) supplied from the image rearranging buffer 6, 
5 thereby generating difference data D5 and forwarding them to 
an orthogonal transform section 6. 

On the contrary, when predictive data D4 is not provided, 
the adder 4 forwards the frame data D2 (I picture) provided from 
the image rearranging buffer 6, as difference data as they are, 

10 to the orthogonal transform section 6. 

The orthogonal transform section 6 carries out an 
orthogonal transf ormprocess , such as discrete cosine transform, 
on the difference data D5 to thereby generate orthogonal 
transform coefficient data D6, and forwards them to a quantizing 

15 section 7. 

The quantizing section 7 carries out a quantizing process 
on the orthogonal transform coefficient data D6 , to thereby 
generate quantized data D7 and forward them to the reverse 
quantizing section 9 and reversible coding section 18. 

2 0 In this case, in the quant izing section 7 , the quantization 

control data D8 for determining a quantization parameter value 
are provided under feedback control by a rate control section 
8, thereby generating quantized data D7 in accordance with a 
quantization parameter value of the quantization control data 

25 D8 - 
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The reversible coding section 18 carries out a reversible 
coding process conforming, for example, to UVCL on the quantized 
data D7 provided from the quantizing section 7 and the motion 
vector D3 provided from the motion-prediction compensating 
5 section 5 , thereby generating coded data D15 and outputting them 
through a storage buffer 19. 

On the other hand, the reverse quantizing section 9 carries 
out a reverse quantizing process on the quantized data D7 provided 
from the quantizing section 7, to thereby restore orthogonal 
10 transform coefficient data D9 corresponding to the orthogonal 
transform coefficient data D6 and forward them to the reverse 
orthogonal transform section 10. 

The reverse orthogonal transform section 10 carries out 
a reverse orthogonal transform process on the orthogonal 
15 transform coefficient data D9 provided from the reverse 
quantizing section 9, to thereby restore difference data DIO 
corresponding to the difference data D5 and forward them to an 
adder 11. 

The adder 11, when provided predictive data D4 from the 
20 motion-prediction compensating section 5, adds a plurality of 
predictive data D4 corresponding to the relevant difference data 
DIO to the difference data DIO provided by the reverse orthogonal 
transform section 11, thereby restoring frame data Dll 
corresponding to the frame data D2 and forwarding them to a deblock 
25 filter 12. 
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On the contrary, the adder 11, when not provided motion 
compensating data D4 from the motion-prediction compensating 
section 5, forwards the difference data DIO provided from the 
reverse orthogonal transform section 11, as frame data Dll as 
5 they are, to the deblock filter 12. 

The deblock filter 12 , when a distortion is caused between 
adjacent macro-blocks of among the frame data Dll provided from 
the adder 11, smoothens the relevant distorted part by filtering 
and stores, as required,, the smoothened frame data Dll, as 

10 reference frame data D12 corresponding to one reference frame 
image, to a frame memory 13. 

On the contrary, the deblock filter 12 , when no distortion 
is caused between adjacent macro-blocks of among the frame data 
Dll, stores as required the frame data Dll, as reference frame 

15 data D12 , to a frame memory 13. 

In this manner, the image coding apparatus 1 is configured 
to store the reference frame data D12, in an amount of data 
corresponding to a plurality of reference frame images to be 
taken as reference candidates, to the frame memory 13. 

20 In addition to the configuration, a first thinning-out 

section 14 thins out by a half horizontally vertically the pixels 
of a reference frame image of the reference frame data D12 read 
from the frame memory 13 , thereby generating a size-reduced image 
having a 1/4 reduction ratio with respect to the reference frame 

25 image as an original image (hereinafter, this is referred to 
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as a 1/4 size-reduced reference image) and storing it as first 
size-reduced data D13 to a 1/4 frame memory 15. 

In this case, the first thinning-out section 14 previously 
stores a FIR filter coefficient value defined in the foregoing 
Equation (1) in its internal memory (not shown) • Using the FIR 
filter coefficient value, the pixels are thinned out according 
toaprocess similar to the motion-compensatingprocess mentioned 
in Fig . 1 . This can generate a 1/4 size-reduced reference image . 

Meanwhile, the first thinning-out section 14 is 
configured, when the reference frame data D12 is an interlaced 
operated image, to make a thinning-out every field with respect 
to a vertical direction as shown in Fig . 6 . For the first field, 
filtering is made by the use of a filter coefficient value given 
by the following equation and previously stored in the internal 
memory . 



For the second field, filtering is made by the use of a filter 
coefficient value given by the following equation previously 
stored in the internal memory . 




0, 88, 138, 88, 0, 




(26) 




7, 



7, 




(27) 



Due to this, pixel thinning-out can be done every other of 
pixel -based values inputted. 

A second thinning-out section 16 previously stores in its 
internal memory a FIR filter coefficient value defined in the 
5 foregoing Equations (1) , (26) and (27) . Similarly to the first 
thinning-out section 14, thinning out is made a half horizontally 
and vertically on the pixels of a 1/4 size-reduced reference 
image of first size-reduced data D13 read out of the 1/4 frame 
memory 15 . Due to this, generated is a size -reduced image having 

10 a 1/16 reduction ratio (hereinafter, this is referred to as a 
1/16 size-reduced reference image) with respect to the reference 
frame image as an original image. This is stored as second 
size-reduced reduced data D14 to the 1/16 frame memory 17. 

In this manner, the image coding apparatus 1 generates 

15 1/4 size-reduced and 1/16 size-reduced images respectively 
having 1/4 and 1/16 reduction ratios on a plurality of reference 
frame images corresponding to a plurality of ones of frame data 
D12 stored in the frame memory 13, in the number corresponding 
respectively to the plurality of reference frame images . These 

20 are stored to the corresponding 1/4 frame memory 15 or 1/16 frame 
memory 17. 

Motion-Prediction Compensating Section Configuration 

Now, explanation is made on the configuration of the 
25 motion-prediction compensating section to whidh the invention 
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is applied. 

As shown in Fig. 7, the motion-prediction compensating 
section 5 as a motion-prediction compensating apparatus is 
configured by' a pixel -block dividing section 20, a size-reduced 
5 block generating section 21 as hierarchizing means, and a 
motion-prediction compensating section 22 as search-range 
determining means and detecting means, to input the frame data 
D2 provided from the image rearranging buffer 3 to the pixel -block 
dividing section 20. 

10 The pixel -block dividing section 2 0 divides the frame image 

of the frame data D2 , based on the block unit having a vertical 
horizontal 16x16 pixel -based size to thereby generate aplurality 
of maximum mot ion compensation blocks , and sequentially forwards 
the maximum motion compensating blocks as maximum motion 

15 compensating block data D20 to the size-reduced block generating 
section 21 and motion-prediction compensating section 22. 

The first size-reduced block generating section 21a of 
the size-reduced block generating section 21 previously stores 
a FIR filter coefficient value as defined in the foregoing 

2 0 Equations (1) , (2 6) and (27) in its internal memory (not shown) , 
and carries out a similar process to the first thinning-out 
section (Fig. 5) thereby thinning out the pixels. 

In this case, the first size-reduced block generating 
section 21a horizontally vertically thins out a half the pixels 

25 of the maximum motion compensating block of the maximum motion 
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compensating block data D2 0 to thereby generate a block having 
a size of vertical/horizontal 8x8 pixels with a 1/4 
size-reduction ratio (hereinafter, this is referred to as a 1/4 
size-reduced block) on the maximum motion compensating block 

5 having a size of vertical /horizontal 16x16 pixels, and forwards 
it as 1/4 size-reduced block data D21 to a second size-reduced 
block generating section 21b and motion-prediction compensating 
section 22 . 

The second size-reduced block generating section 21b 
10 previously stores a FIR filter coefficient value as defined in 
the foregoing Equations (1), (26) and (27) in its internal memory 
(not shown) , and carries out a similar process to the second 
thinning-out section (Fig. 5) thereby thinning out the pixels. 

In this case, the second size-reduced block generating 
15 section 21b horizontally vertically thins out a half the pixels 
of the 1/4 size-reduced block of the 1/4 size-reduced block data 
D21 to thereby generate a block having a size of 
vertical/horizontal 4x4 pixels with a 1/16 size-reduced ratio 
(hereinafter, this is referred to as a 1/16 size-reduced block) 
2 0 on the maximum motion compensating block having a size of 
vertically horizontally 16x16 pixels, and forwards it as 1/16 
size-reduced block data D22 to the motion-prediction 
compensating section 22. 

The motion-prediction compensating section 22 
2 5 recognizes, as an original image group on the uppermost layer 



TS, the maximum motion compensating block UBl corresponding to 
the maximum motion compensation block data D2 0 provided from 
the pixel -block dividing section 2 0 and the reference frame 
images SFl - SF4 corresponding to, for example, four kinds of 
5 reference frame data D12 read out of the frame memory 13, as 
shown in Fig . 8 . 

Meanwhile, the motion-prediction compensating section 22 
recognizes, as a size -reduced image group on the first lower 
layer BSf , the 1/4 size-reduced block CBf corresponding to the 

10 1/4 size-reduced block data D21 provided from the first 
size-reduced block generating section 21a and the 1/4 
size-reduced reference frame images CBl - CB4 corresponding to, 
for example, four kinds of first size-reduced image data D13 
read out of the 1/4 frame memory 15. 

15 Furthermore, the motion-prediction compensating section 

22 recognizes, as a size -reduced image group on the lowermost 
second layer BSs, the 1/16 size-reduced block CBs corresponding 
to the 1/16 size-reduced block data D22 provided from the second 
size-reduced block generating section 21b and the 1/16 

2 0 size-reduced reference frame images CBll - CB14 corresponding 
to the second size-reduced image data D14 read out of the 1/16 
frame memory 17. 

Thus, the motion-prediction compensating section 22 
carries out a motion-prediction compensating process by the use 

2 5 of the recognized image groups of the uppermost layer TS, first 
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lower layer BSf and second lower layer BSs to thereby generate 
a motion vector D3 and predictive data D4 , and forwards the motion 
vector D3 to the reversible coding section 18 and the predictive 
data D4 to the adders 4 and 11. 
5 In this manner, the motion-prediction compensating 

section 5 carries out a motion-prediction compensating process 
with separation to the hierarchical layers TS, BSf and BSs, by 
taking a set of the maximum motion compensating block UBl, 1/4 
size-reduced block CBf and 1/16 size-reduced block CBs and the 

10 reference frame image SFl - SF4, 1/4 size-reduced reference image 
CBl - CB4 and 1/16 size-reduced reference image CBll - CB14 
respectively corresponding to the reduction ratios of the maximum 
motion compensating block UBl, 1/4 size-reduced block CBf and 
1/16 size-reduced block CBs. 

15 Meanwhile, the motion-prediction compensating section 22 

is configured to carry out a motion-prediction compensating 
process by a control section configured by a not -shown CPU, ROM, 
RAM and the like, according to a predetermined motion-prediction 
compensating program previously stored in the ROM. The 

2 0 motion-prediction compensating process procedure is explained 
below. 

Motion-prediction Compensating Process Procedure 

As shown in Fig. 9, the motion-prediction compensating 
25 section 22 moves from the start step of a routine RTl to the 
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following step SPl, to sequentially make a block matching on 
the respective ones of 1/16 size-reduced reference images CBll 
- CB14 on the lowermost, second lower layer BSs (Fig. 8) with 
a 1/16 size-reduced block CBs having a size of 
5 vertical/horizontal 4x4 pixels, thereby searching for a motion 
vector on the second lower layer BSs (hereinafter, this is 
referred to as a second lower layer motion vector) . 

Specifically, the motion-prediction compensating section 
22 determines only the absolute-value total sums of differences 

10 between every pixel-based value within the 1/16 size-reduced 
block CBs and the pixel-based value of a block within the 1/16 
size -reduced reference image CBll (CB12 - CB14) corresponding 
to that block CBs (hereinafter, this is referred to as a 
corresponding block) without applying RD optimization, for 

15 example, as shown in Fig. 10. Based on a motion amount between 
the corresponding block CTB of among those the absolute -value 
total sum assumes the minimum and the 1/16 size -reduced block 
CBs, detected is a second lower layer motion vector 1/16MV1 
(l/16iyiV2 - l/l6iyiV4) on the second lower layer BSs (Fig. 8) . Then, 

20 the process moves to the next step SP2 . 

In this case, the motion-prediction compensating section 
22 makes a block matching on the 1/16 size-reduced reference 
image CBll - CB14 smaller in search range as compared to the 
reference frame image SFl - SF4 as an original image without 

25 making reference to a generation code amount table representative 
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of a generation code amount of a motion vector difference and 
without carrying out an operation of quantizing function 
(Lagrange multiplier on the motion vector) . This can reduce 
the process burden upon determining the second lower layer motion 
5 vectors 1/16MV1 - 1/16MV4. 

In the step SP2 , the motion-prediction compensating 
section 22 makes a motion compensation process (interpolation 
process) on the horizontal and vertical components of the second 
lower layer motion vector 1/16MV1 - 1/16MV4 detected in the step 

10 SPl as shown for example in Fig. 11, thereby enlarging the 
horizontal component twice and the vertical component twice, 
i.e. four times in total - 

In this case, the motion-prediction compensating section 
22, even when the 1/4-pixel accuracy (Fig, 1) or 1/8-pixel 

15 accuracy (Fig. 2) under the JVT coding scheme is selected for 
the maximum motion compensating block UBl on an original image 
corresponding to the 1/16 size-reduced block CBs , makes amotion 
compensation according to an integer-pixel accuracy or 1/2 -pixel 
accuracy based on linear interpolation- This can reduce the 

2 0 process burden during the motion compensation. 

The motion-prediction compensating section 22 determines 
the range UARl (UAR2 - UAR4) including the peripheral several 
pixels in a rectangular range AR having apexes of start and end 
points of the second lower layer motion vector enlarged by the 

25 motion compensation, as a search range within a 1/4 size-reduced 
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reference image CBl (CB2 - CB4) on the first lower layer BSf 
(hereinafter, this is referred to as a first lower layer search 
range) . The process moves to the next step SP3 . 

In the step SP3, the motion-prediction compensating 
5 section 22 sequentially makes a block matching on the first lower 
layer search ranges UARl - UAR4 within the 1/4 size-reduced 
reference images CBl - CB4 determined in the step SP2 with a 
1/4 size-reduced block CBf having a size of vertical/horizontal 
8x8 pixels instead of applying RD optimization similarly to the 

10 process in the step SPl, thereby detecting motion vectors 1/4MV1 
- 1/4MV4 (Fig. 8) (hereinafter, this is referred to as a first 
lower layer motion vector) in the first lower layer . The process 
moves to the next step SP4 . 

In the step SP4, the motion-prediction compensating 

15 section 22 carries out a similar process to the process in the 
step SP2 on the first lower layer motion vectors 1/4MV1 - l/4iyiV4 
detected in the step SP3 , thereby determining a search range 
(hereinafter, this is referred to as an uppermost layer search 
range) TARl - TAR4 within the reference frame image SFl - SF4 

20 (Fig. 8) on the uppermost layer TS . The process moves to the 
next step SP5 . 

In the step SP5, the motion-prediction compensating 
section 22 sequentially makes a block matching by RD optimization 
on the first uppermost layer search ranges TARl - TAR4 (Fig. 

2 5 8) determined in step SP4 , with a motion compensating block UBl 
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having a size of vertical/horizontal 16x16 pixels as defined 
under the JVT coding scheme, thereby detecting a motion vector 
related to the motion compensating block UBl . The process moves 
to the next step SP6 . 
5 In this case, the motion-prediction compensating section 

22 searches only within the first uppermost layer search range 
TARl - TAR4 and detects a motion vector related to the motion 
compensating block UBl without searching for a motion vector 
throughout the entire maximum motion compensating block as 

10 defined under the JVT coding scheme, thereby making it possible 
to greatly reduce the process burden upon RD optimization 
requiring a huge operation amount as noted before. 

In step SP6, the motion-prediction compensating section 
22 uses the first uppermost layer search range TARl - TAR4 on 

15 the uppermost layer TS (Fig. 8) , to carry out a block matching 
by RD optimization on each motion compensating block UB2 - UB34 
while sequentially changing from the greater pixel -based size 
to the smaller pixel-based size based on each of the first to 
third series SSI - SS3 whose motion compensating blocks are the 

20 same in block form as shown in Fig. 12. 

Inactuality, the motion-prediction compensating section 
22, in the first series SSI, carries out a similar process to 
the search range determination noted in Fig. 11 (hereinafter, 
this is referred to as a search range determining process) 

25 depending upon a motion vector of motion compensating block UBl 
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detected in the step SP5, to determine a second uppermost layer 
search range TARS - TARS smaller than the first uppermost layer 
search range TARl - TAR4 (Fig. 8) as shown, for example, in Fig. 
13 . Withinonly the relevant second uppermost layer search range 
5 TARS - TARS, detected are two kinds of motion vectors related 
to the motion compensating blocks UB2 and UB3 each having a size 

of vertical/horizontal 8x16 pixels. 

Then, the motion-prediction compensating section 22 
carries out a search- range determining process depending upon 

10 a motion vector related to the motion compensating block UB2 
and UB3 , to determine a third uppermost layer search range TAR9 
- TAR12 smaller than the second uppermost layer search range 
TARS - TARS . Within only the relevant third uppermost layer 
search range TAR9 - TAR12, detected are four kinds of motion 

15 vectors related to the motion compensating blocks UBll - UB14 
each having a size of vertical/horizontal 4x8 pixels. 

Meanwhile, the motion-prediction compensating section 
22 , in the second series SS2 , detects two kinds of motion vectors 
related to the motion compensating blocks UB4 and UBS each having 

20 a size of vertical/horizontal 16x8 pixels only within the second 
uppermost layer search range TAR S - TARS similarly to the process 
on the first series SSI, nearly simultaneously to the process 
on the first series SSI. Thereafter, within only the relevant 
third uppermost layer search range TAR9 - TAR12, detected are 

2S four kinds of motion vectors related to the motion compensating 
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blocks UB15 - UB18 each having a size of vertical/horizontal 
8x4 pixels - 

Furthermore, the motion-prediction compensating section 
22 , in the third series SS3 , detects four kinds of motion vectors 
5 related to the motion compensating blocks UB6 - UB9 each having 
a size of vertical/horizontal 8x8 pixels only within the second 
uppermost layer search range TAR5 - TARS similarly to the process 
on the first series SSI, nearly simultaneously to the process 
on the first series SSI. Thereafter, within only the relevant 

10 third uppermost layer search range TAR9 - TAR12, detected are 
motion vectors related to the motion compensating blocks UB19 
- UB34 each having a size of vertical/horizontal 4x4 pixels. 

In this manner, the motion-prediction compensating 
section 22 reduces its search range each time sequentially 

15 changing to a motion compensating block of a smaller pixel -based 
size, thereby making possible to reduce the process burden upon 
detecting a motion vector in a smaller pixel-based size of motion 
compensating block requiring a greater operation amount by an 
increase of the number of blocks as compared to a motion 

20 compensating block greater in pixel-based size. Furthermore, 
process burden can be greatly reduced as compared to the case 
of determining a motion vector in each pixel-based size of motion 
compensating block (Fig. 16) throughout the entire macro-block 
as defined under the JVT coding scheme. 

25 In this manner, the motion-prediction compensating 
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section 22 detects motion vectors respectively related to the 
remaining motion compensating blocks UB2 - UB34 other than the 
motion compensating block UBl. Thereafter, the process moves 
to the next step SP7 . 
5 In this connection, the motion-prediction compensating 

section 22, in the first to third series SSI - SS3 , used the 
same one of the second uppermost layer search range TARS - TARS 
and third uppermost layer search range TAR9 - TAR12 . However, 
this is notlimiting, i.e., in case the search range is reduced 
10 each time sequentially changing to the smaller pixel -based size 
of motion compensating block, the form or the like may be 
different . 

In the step SP7, the motion-prediction compensating 
section 22 detects as the final motion vector D3 (Fig. 7) an 
15 optimal motion vector capable of generating the smallest code 
amount among the motion vectors of the motion compensating blocks 
UBl - UB34 respectively detected by RD optimization in the steps 
SP5 and SP6 . The process moves to the next step SP8 . 

In the step SP8, the motion-prediction compensating 
20 section 22 carries out a 1/4 motion compensating process noted 
before for example in Fig. 1 according to the motion vector D3 
detected in the step SP7 , thereby generating predictive data 
D4 (Fig. 7) . The process moves to the next step SP9. 

In the step SP9, the motion-prediction compensating 
25 section 22 forwards the motion vector D3 detected in the step 
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SP7 to the reversible coding section 18 and the predictive data 
D4 generated in the step SP8 to the adders 4 and 11 . The process 
moves to the next step SPIO, to end the motion-prediction 
compensating process. 

5 

Operation and Effect 

In the above configuration, the motion-prediction 
compensating section 5 takes as the uppermost layer TS a motion 
compensating block UBl (Fig. 8) having the greatest pixel -based 

10 size among the motion compensating blocks defined under the JVT 
coding scheme. By thinning out over the maximum motion 
compensating block UBl, generated and hierarchized are a 
size-reduced block CBs on first lower layer having a 1/4 
size-reduction ratio and a 1/16 size-reduced block CBf on second 

15 lower layer having a 1/16 size-reduction ratio. 

Then, the motion-prediction compensating section 5 uses 
a plurality of reduced reference images CFll - CF14 and CFl - 
CF4 of the lower layers (first lower layer BSf and second lower 
layer BSs) excepting the uppermost layer TS and the reduced blocks 

20 CBs and CBf, to hierarchically search for motion vectors and 
determine a first uppermost layer search range TARl - TAR4 for 
motion vector search by RD- optimization on the uppermost layer 
TS (steps SPl - SP4) . 

In this case, the motion-prediction compensating section 

25 5 detects a motion vector by carrying out a block matching based 
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only on the absolute-value total sum of the difference from the 
size-reduced block without applying RD optimization which takes 
into account a generated code amount representative of a 
generated code amount of motion vector difference and a 
5 quantizing function, and makes a mot ion- compensation on the 
detected motion vector by linear interpolation at integer-pixel 
accuracy or 1/2 -pixel accuracy without making a 
motion-compensation at 1/4 -pixel accuracy (Fig. 1) or 1/8-pixel 
accuracy (Fig. 2) . Thus, a first uppermost layer search range 

10 TARl - TAR4 can be determined by a reduced amount of operation. 

In this state, the motion-prediction compensating section 
5 detects motion vectors related respectively to the motion 
compensating blocks UBl - UB34 (Fig. 12) having respective 
pixel -based sizes by the use of the first uppermost layer search 

15 range TARl - TAR4 . Among the detected motion vectors, detected 
is the optimal and final motion vector D3 capable of generating 
the smallest amount of code. 

Accordingly, despite the motion-prediction compensating 
section 5 is added by the new operation in an amount required 

2 0 in the process of up to determining the first uppermost layer 
search range TARl - TAR4 , restricted is the search range for 
RD optimization requiring by far greater operation amount than 
that operation amount. Correspondingly, process burden can be 
greatly reduced as compared to the case to detect a motion vector 

2 5 by searching for a motion vector throughout the entire 
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macro-block on each mot ion compensat ing block in each pixel -based 
size as defined under the JVT coding scheme. 

In addition, the motion-prediction compensating section 
5, as noted above in Fig. 12, whendetectingmotion vectors related 
5 to the motion compensating blocks UBl - UB34 (Fig. 12) in the 
respective pixel -based sizes, reduces the search range each time 
of sequentially changing the motion compensating blocks from 
the greater pixel-based size to the smaller pixel-based size. 
This can reduce the process burden upon detecting a motion vector 
10 on a motion compensating block in a smaller pixel-based size. 
As a result, the overall process burden can be reduced 
furthermore . 

According to the above configuration, the motion 
compensating block UBl in the greatest pixel -based size is 

15 hierarchized. Using a plurality of reduced reference images 
CFll - CF14 and CFl - CF4 in the hierarchized first lower layer 
BSf and second lower layer BSs as well as reduced blocks CBs 
and CBf , the optimal motion vector in the uppermost layer TS 
is narrowed down to roughly determine a first uppermost layer 

2 0 search range TARl - TAR4 by a reduced amount of operation. 
Thereafter, using the relevant search range TARl - TAR4, a motion 
vector is detected by RD optimization. This can greatly reduce 
the process burden upon detecting the motion vector D3 . 

2 5 Other Embodiment 
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Incidentally, the above embodiment described the case for 
carrying out a motion-prediction compensation by sequentially 
changing to the respective motion compensating blocks having 
respective pixel-based sizes of vertical/horizontal 16x16 
5 pixels, vertical/horizontal 8x16 pixels, vertical/horizontal 
16x8 pixels, vertical/horizontal 8x8 , pixels, 

vertical/horizontal 4x8 pixels, vertical/horizontal 8x4 pixels 
and vertical/horizontal 8x4 pixels. However, the invention is 
not limited to this but motion-prediction compensation may be 
10 made by sequentially changing to the motion compensating blocks 
in other various pixel -based sizes, e.g. , vertical/horizontal 

16x4 pixel-based size. 

Furthermore , although the above embodiment described the 
case that the invention is applied to the JVT coding scheme, 

15 this is not limitative. In brief, the invention can be applied 
to the other various coding schemes that motion-prediction 
compensation is made by using a plurality of reference frame 
images while sequentially changing the pixel-based size of the 
motion compensating block on each motion compensating block that 

2 0 the objective frame image to be processed is divided of the 
successive frame images . 

Furthermore, although the above embodiment described the 
case that thinning-out is made using Equation (1) taking as a 
FIR filter coefficient, the invention is not limited to this 

2 5 but other various FIR filter coefficient values may be used to 
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generate a 1/2 phase. 

Furthermore, although the above embodiment described the 
case that reduction is at the same reduction ratio in the 
horizontal and vertical directions, the invention is not limited 
5 to this but size-reduction may be at different size-reduction 
ratios between the horizontal direction and vertical direction. 

Furthermore, although the above embodiment described the 
case that size-reduction is at 1/4 and 1/16 size-reduction 
ratios, the invention is mot limited to this, i.e., 

10 size -reduction may be at the other reduction ratio, e.g., lover 
the power of 2 . 

Furthermore, the above embodiment described the case to 
determine amotion vector depending upon an absolute-value total 
sum of differences between every pixel -based value within a 1/ 16 

15 size-reduced block CBs and a pixel -based value of the 
corresponding block within the 1/16 size-reduced reference image 
CBll - CB14 corresponding to the relevant block CBs, or between 
every pixel -based value within a 1/4 size-reduced block CBf and 
a pixel -based value of the corresponding block within the 1/4 

2 0 size-reduced reference image CBl - CB4 corresponding to the 
relevant block CBf. However, the invention is not limited to 
this but a square sum of the differences may be determined. In 
brief, motion vectors can be determined on the basis of the 
difference values between the size-reduced block pixel -based 

25 value and the pixel-based value of reduced reference image. 
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Furthermore, the above embodiment described the case, as 
noted above in Fig. 8, of using every reference size-reduced 
image CFll - CF14 and CFl - CF4 in the second lower layer BSs 
and first lower layer BSf as well as size-reduced blocks CBs 
5 and CBf , to determine the first uppermost layer search range 
TARl - TAR4 on every reference frame image SFl - SF4 in the 
uppermost layer TS . However, the invention is not limited to 
this but may determine any of the uppermost layer search ranges 
TARl - TAR4 . 

10 In this case, following the process of the step SPl in 

the motion-prediction compensating process procedure mentioned 
in Fig. 9, added is a step of selecting, among reference 
size-reduced images CFll - CF14 (Fig. 8) , a reference 
size-reduced image CF13, for example, that the difference of- 

15 pixel -based value is minimum from the reduced block CBs by a 
block matching with a size-reduced block CBs on the second lower 
layer BSs. Using only the reference size-reduced image CF3 on 
first lower layer BSf corresponding to the reference size-reduced 
image CF13 selected in that step, the motion-prediction 

20 compensating section 22 carries out the processes of steps SP2 
to SP4 , to thereby determine only the first uppermost layer search 
range TAR3 . 

In this manner, the motion-prediction compensating 
section 22, in the second lower layer BSs, selects a reference 
25 frame image SF3 for RD-optimization mot ion- vector search on the 
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uppermost layer TS . This can reduce the respective amounts of 
operation corresponding to a plurality of reference size -reduced 
images CFll, CF13, CF14 , CFl, CF3 and CF4 and a plurality of 
reference frame images SFl, SF3 and SF4 . The overall process 
5 burden can be greatly reduced of up to detecting the final optimal 
motion vector D3 (Fig. 7) . 

Furthermore, where the motion compensating block UBl to 
be processed is of a picture, the motion-prediction compensating 
section 22 in the second lower layer BSs may take, by a block 

10 matching with the reduced block CBs , a predictive-direction mode 
having a motion vector the pixel -based value difference is 
minimum from the size -reduced block CBs as a predictive -direct ion 
mode for the motion compensating block UBl among the reference 
reduced images CFll - CF14 (Fig. 8) , similarly to the foregoing 

15 selection of a reference frame image SF3 . By doing so, the 
overall process burden can be further reduced of up to detecting 
the final optimal motion vector D3 (Fig. 7) . 

Furthermore , the above embodiment described the case that , 
as described in Fig. 11 as an example of determination of search 

2 0 ranges for example, the motion vector detected on the second 
lower layer is enlarged to determine as a first lower layer search 
range a range UARl including the peripheral several pixels of 
a rectangular range AR having, as apexes, the start and end points 
of that enlarged motion vector. However, the invention is not 

25 limited to this. In brief, the peripheral pixel -based range 
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including an enlarged motion vector can determine a search range , 
e.g. trapezoidal range, rectangular-parallelepiped range, 
circular range including peripheral pixels having as apexes start 
and end points of that enlarged motion vector. 
5 Furthermore, the above embodiment described the case to 

detect the optimal motion vectors for the motion compensating 
blocks UB2 - UB34 in the order mentioned in Fig. 12. However, 
the invention is not limited to this but may detect the optimal 
motion vectors for the motion compensating blocks UB2 - UB34 

10 in the order as in Fig . 14 shown attached with the same references 
to the corresponding parts to Fig. 12. 

Furthermore, the above embodiment described the case to 
make a block matching by the use of all the pixel -based values 
within the 1/16 size-reduced block CBs, 1/4 size-reduced block 

15 CBf or maximum motion compensating block UBl . However, the 
invention is not limited to this but may make a block matching 
by the use of every other pixel -based value with respect to each 
of the horizontal and vertical directions within the 1/16 
size-reduced block CBs, 1/4 size-reduced block CBf or maximum 

20 motion compensating block UBl as shown in Fig. 15. In this case, 
the process burden of upon carrying out a block matching can 
be reduced furthermore. 

Furthermore, the above embodiment described the case to 
realize the processes in the motion-prediction compensating 

25 section 22 by a motion compensating program. The invention is 
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not limited to this but the processes in part or the entire may 
be realized on the hardware means of exclusive integrated 
circuits or the like. 

Furthermore, the above embodiment described the case to 
5 realize a size -reducing process by the hardware -configured 
size-reduced block generating section 21 (Fig. 7) , However, 
the invention is not limited to this but the size-reducing process 
in the size-reduced block generating section 21 may be realized 
by being built in a motion-prediction compensating program in 

10 the motion-prediction compensating section 22. 

Furthermore, the above embodiment described the case for 
the motion-prediction compensating section 5 as a 
motion-prediction compensator to implement a motion-prediction 
compensation by expanding in the internal RAM (not shown) a 

15 motion-prediction compensating program stored in the internal 
ROM (not shown) according to the foregoing motion-prediction 
compensating process of Fig. 9. The invention is not limited 
to this but motion-prediction compensation may be implemented 
by installing a program medium storing the motion-prediction 

2 0 compensating program. 

The program medium may be realized by a semiconductor 
memory, a magnetic disk or the like, besides a packaged media 
such as a CD-ROM (Compact Disc-Read Only Memory) or DVD (Digital 
Versatile Disc) . Meanwhile, the means for storing a program 

25 to the program medium may utilize a wired or wireless radio 
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communicating medium, such as local area networks, the Internet 
or digital satellite broadcast . Storage may be through various 
interfaces, such as routers and MODEMS. 

As described above, according to the present invention, 
5 when compensating for motion prediction on each of motion 
compensating blocks formed by dividing an objective frame image 
of successive frame images by using a plurality of reference 
frame images while sequentially changing pixel -based sizes of 
the motion compensating blocks, pixels are thinned out of the 

10 motion compensating block having a greatest one of the 
pixel -based size to be taken as an uppermost layer of among the 
pixel -based sizes, to generate and hierarchize a size-reduced 
block in a lower layer having a predetermined size -reduction 
ratio. Thereafter, motion vector search ranges are determined 

15 respectively within the reference frame images, on the basis 
of the size-reduced blocks and a plurality of size-reduced 
reference images size -reduced corresponding respectively to the 
size-reduction ratios of the size-reduced blocks. An optimal 
one of the motion vectors is detected while sequentially changing 

20 the pixel-based sizes of the motion compensating blocks by using 
the search ranges determined - 

In this case, motion vectors can be detected only in a 
limited search range of the reference frame image without using 
the entire of the reference frame image . Thus , the process burden 

2 5 upon code processing can be reduced while possibly avoiding the 
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lowering of image quality. 

The present patent application contains subject matter 
related to that disclosed in Japanese Priority Document 
JP2002-350138, filed in the JPO on December 2 , 2002, the entire 
contents of which being incorporated herewith by reference. 



